In recent years, video communication over network environment, such as Internet, has been an important application. For example, video conferencing can be conducted over various networks. In network transmission, the bandwidth of multimedia data should be stable in order to keep transmission smooth. Nevertheless, for a video bitstream, the bitstream size of each frame varies according to various factors, such as frame type, frame contents, and quantization parameter (QP). For the frame type, it may be an Intra coded frame or Inter coded frame, where an Intra coded frame often generated much a high bit count than a typical Inter coded frame. For frame contents, high-frequency frame contents often result in a higher bit count than low-frequency frame contents. For quantization parameter, a larger quantization parameter will result in higher video quality at the expense of a higher bit count. Furthermore, the bit rate is also related to coding efficiency of the video standard applied.
In video coding, Intra coded frame usually is used once a while or whenever needed. For example, Intra frame may be sent periodically (e.g. every 30 frames) or when a scene change occurs. An Intra coded frame will induces large fluctuation of network bandwidth. Accordingly, Intra coded frame is not favored and the number of Intra coded frame should be reduced.
For compressed video, the video quality suffers from the network transmission errors due to the use of Inter frame coding and the use of variable length coding. The Inter frame coding will cause transmission errors to propagate from frame to frame. Therefore, the use of Intra coded frame can constrain the error propagation due to Inter frame coding. On the other hand, any error in the bitstream may cause variable length codes to lose sync.
Since the Intra coded frames often result in a higher data rate, the mechanism of sending Intra coded frame has to be controlled carefully in order to avoid possible visual quality degradation due to network transmission issues. In order to address the issue, various multimedia transport technologies have been developed. For example, in IETF RFC 4585 (J. Ott et al., Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF), Internet Engineering Task Force, Request for Comments: 4585, July 2006), an extension to the Audio-visual Profile (AVP) is defined that enables receivers to provide, statistically, more immediate feedback to the senders. Thus the technique allows for short-term adaptation and efficient feedback-based repair mechanisms to be implemented. Furthermore, IETF RFC 5104 (S. Wenger et al., Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF), Internet Engineering Task Force, Request for Comments: 5104, February 2008) specifies a few extensions to the messages defined in the Audio-Visual Profile with Feedback (AVPF), which are helpful primarily in conversational multimedia scenarios, where centralized multipoint functionalities are used.
AVPF allows feedback information during a video call. AVPF can be used for Real-time Transport Control Protocol (RTCP), also called RTP/AVPF. The AVPF protocol results in low latency feedback. The feedback messages from a decoder to an encoder include Full Intra Request (FIR) that requests the media sender to send a Decoder Refresh Point at the earliest opportunity. The feedback information also includes Payload-Specific Feedback Messages such as Picture Loss Indication (PLI) and Slice Loss Indication (SLI). For PLI, a decoder may inform an encoder about the loss of an undefined amount of coded video data belonging to one or more pictures by signaling PLI. For SLI, a decoder may inform an encoder that it has detected the loss or corruption of one or several consecutive macroblock(s) in a scan order.
For the feedback message, it has transmission latency between the transmitter device and the receiver device. On the transmitter side, the transmitter device transmits the feedback message to the receiver device. At the receiver side, the receiver device receives the feedback message from the transmitter device. If the transmitter device transmits the first feedback message at time T1 and the receiver device receives the first feedback message at time T2, then the relation of time T1 and T2 should satisfy the condition, T1<T2. In other words, the receiver device receives the first feedback message after the transmitter device transmits the first feedback message. The transmission latency is determined according to (T2−T1).